BBS Toolkit/BBS Toolkit.iso/wildcat/dpmsg261.zip/DUPMSG.DOC

Text File | 1993-04-28 | 23KB | 565 lines

┌──────────────────────────────────────────┐
│ DupMsg.exe     v2.61           04-26-93  │
└──────────────────────────────────────────┘

What it Does
------------

DupMsg finds and deletes duplicate messages from Wildcat 3+
message databases.  Duplicates will be identified even if the
tagline, date or subject line has been changed.  In addition,
DupMsg will delete old messages by date, purge messages over the
MakeWild limit, and delete messages from specific users as desired.

Deleted messages can be completely purged from the database or
internally marked as "deleted" for later review.  Marked messages
can be purged on a subsequent DupMsg run.

Since some BBS utilities modify the subject line and the message's
date and time, DupMsg can be configured to ignore both subject and
date/time when identifying duplicates.  PC Relay users should
select the "ignore date/time" option as the PC Relay program does
change the message's date.


How it Works
------------

Upon opening a database, DupMsg first purges messages that were
marked for later deletion.  It uses the marked-for-deletion key
from the index key file (MSGxxx.IX) to identify these messages.

The database is then searched for messages from users listed in the
DUPMSG.LST file.  This process uses a key search so it is quite
fast, unless there are many names in the DUPMSG.LST file.

Next, messages from one person at a time are read and each
message's date is checked to see if it is too old.  Old messages
are deleted on this first scan through the database.  Suspected
duplicates are then compared text vs. text and duplicates are
deleted.  If a duplicate pair is found, the message with the higher
message number is deleted.

Finally, if the number of messages still exceeds the limit set in
MakeWild, the excess messages will be purged from the database
starting with the lowest message number.  This process is automatic
and is useful when you reduce the number of messages allowed per
conference or use WildMail's no-purge option.  If the MakeWild
limit is set to zero (0), then this option is disabled for DupMsg
and all other message import utilities.

The standard tagline format has a space as the first character in
the line, then either an asterisk or square box as the second
character.  DupMsg recognizes the standard formats as well as non-
standard taglines from the following programs:
     PC Relay 4.1x
     SF UTI   3.16
     RBBS-UTI 2.0

DupMsg will also handle FidoNet ^@ control lines and word wrapping.
While the DupMsg author will try to keep up with different non-
standard taglines, the standard format should be used in programs
whenever possible.  Standard formats will allow all programs to
work more efficiently.


Other Information
-----------------

If DupMsg deletes a message with an attached file, the attached
file will be deleted as well.  Text files that are imported using
the <<*.IMP method will not be deleted.

Pressing CTRL-C will signal DupMsg to stop processing at the next
safe opportunity, save and close the database, and update the
current message counter.

If running DupMsg remotely using the Doorway program, using the
/V:D option to handle direct screen writing.

Messages that are marked with the "No Delete" flag will not be
deleted.

DupMsg treats slow message conferences a little differently.  If
there are fewer than 10 messages in a conference, they will not be
deleted because of their age;  Duplicates, marked and blocked
messages will still be deleted.  In the future, this option will be
configurable.


Setup
-----

DupMsg can be run from any directory and looks for its
configuration file and the required WildCat files in the path.  As
long as the main WildCat directory is in the path, you should have
no problems with DupMsg.

DupMsg uses the following files:

     MAKEWILD.DAT        Found using the WCHOME variable, in the
                         current directory, or in the path.
     CONFDESC.DAT        In the same directory as MAKEWILD.DAT.
     NODEINFO.DAT        Path is defined in Makewild.

     DUPMSG.CFG          Either found in the current directory or
                         path or will be created by DupMsg's Setup
                         section.

     DUPMSG.LST          Found in the current directory or in the
                         path.

     DUPMSG.LOG          If found in the current directory or path,
                         all log entries will be added to the
                         current file.  If not found, then a new
                         log file will be created in the current
                         directory.

     All the messages conferences are defined using Makewild and
     the descriptions are stored in Confdesc.dat.

The first time DupMsg is used, it will run the configuration module
and create a config file.  The config file allows you to customize
the DupMsg program to meet your needs.  The config file options
are:


                         Master Settings


1)  Log Options

     No Log:   No information is written to the log or echoed to
               the screen.  If all three fields are set to "No
               Log", then no log file will be created.

     Brief:    Displays and logs the message number being deleted
               and the reason for deletion.

     Complete: Displays the above information and a second line
               showing the message FROM-name, TO-name, and
               Subject.  For duplicate messages, the message's
               source (uploader) will be noted, if it can be
               determined.

2)  Delete Options

     Delete:   Completely delete messages from the database and
               free up the space for new messages.  The drawback
               is that deleted messages cannot be recovered; once
               deleted they are gone forever.

     Mark:     Messages are not removed from the database and
               still occupy space on the disk.  Marked messages
               can be reviewed by the Sysop, undeleted, or
               completely deleted at a later time.

     * OFF *:  Turns this function off for all conferences.

3)  Duplicate Options

     Ignore Date:  Some mail doors replace the date and time of a
               message with the date and time that the message was
               uploaded.  The same message uploaded twice may
               cause a pair of duplicates to have different dates.
               If YES, duplicate messages with different dates
               and/or times will be deleted.  If NO, they will be
               considered different messages.

     Ignore Subject:  If YES, duplicate message with different
               subjects will be deleted.  This can be useful to
               catch duplicate BBS and For Sale advertising.  If
               NO, they will be considered different messages.

               Some BBS utilities change a mixed case subject line
               to all upper case.  Because of this, upper and
               lower case differences are always ignored.


                    Conference Configuration

1)   Purge marked messages:  All messages marked as deleted will be
     purged from the database.  This option is performed before any
     duplicate or old messages are marked, so only messages marked
     directly from the WildCat message menu or on prior DupMsg runs
     will be purged from the database.

2)   Delete blocked messages:  This refers to messages from users
     who are listed in the DUPMSG.LST file as being blocked from
     your BBS.  All messages FROM users in your DUPMSG.LST file
     will be deleted.

3)   Delete duplicate messages:  The most important switch in the
     program, this turns duplicate message checking on or off for
     each conference.  Exact duplicates and messages with extra &
     different taglines will be found and the newer message (the
     one with the higher message number) will be deleted.

4)   Delete old messages:  Messages older than the number of days
     specified will be deleted.  Old messages are determined based
     on the date & time of the messages, not on when the messages
     were introduced to the BBS.  If a messages has the "No Delete"
     flag set (see WildCat docs) then that message will not be
     deleted.

5)   Number of days:  If "Delete Old Messages" is set to "YES",
     this is the number of days that a message will be kept in the
     database.  The default is 30 days, so messages over 30 days
     old will be deleted, even if they have just been uploaded to
     the BBS.  If this field is set to zero, then all messages will
     be deleted, except those with today's date.

6)   High message number:  Each time the program is run, DupMsg
     stores each conference's high message number in this field.
     If the current high message number is the same as the number
     stored in the config file, then no duplicate checking is
     performed.  If the current number is larger, then only the new
     messages will be checked as possible duplicates.  If the
     current number is smaller than the config file number, then
     DupMsg will assume that the database has been renumbered and
     will check every message.


Blocked Messages  (DUPMSG.LST)
------------------------------
You now have the option of deleting all messages from a given user
name.  When configured to delete blocked messages, DupMsg will read
each line from DUPMSG.LST and search for all messages FROM that
user name.  If found, the message(s) will be marked or deleted as
configured.

This is useful for problem users on the network that you cannot
lock out any other way, but it should be used sparingly.  Examples
of proper use are:

     --   A user consistently uses foul language, and the source
          BBS is not taking care of it.
     --   A chain letter, advertisement, etc. was left in every
          single message conference.

There is a danger that this option may be used for censorship, or
that innocent users may be locked out unnecessarily.  Sysops should
take care to use it only when necessary.

The DUPMSG.LST file is a standard text file that contains one line
per user name.  Case is ignored when processing the file.


Memory Requirements
-------------------

DupMsg requires a minimum of 198K to run, but will use up to 255K
if available.  Using maximum memory, DupMsg is able to compare up
to 2000 messages at a time.  Since many BBSs have conferences with
more than 2000 messages, DupMsg selects and processes messages from
only one person at a time.

If there are more than 2000 messages from any one user, or if the
program runs out of memory, DupMsg will process the first 2000
messages or as many as it can hold in memory, then go back and
continue duplicate checking from where it left off.  Duplicate
messages will only be missed if one copy is in the first 2000 and
the other copy is in subsequent batches.

Remember, the limitation is 2000 messages from a single user, not
2000 total messages.  DupMsg only checks for duplicates in one
conference at a time.


Multi-Node BBSs
---------------
If you are running a single-line version of WildCat, DupMsg cannot
be run while the BBS is up, or when any other utility is accessing
the message databases.

In order to run DupMsg while the BBS is online, you must be running
a multi-line WildCat version and have either DOS share or Novell
defined in the Makewild program.  In addition, if the autonode
option is not selected in MAKEWILD, the environment variable
WCNODEID must be set to a valid number between 1 and 250.

Use the following command:

     SET WCNODEID=3

using 3 or whichever node ID you wish to use.  This node ID must
not be duplicated.  No other BBS node or program using this node ID
may be running while DupMsg is accessing the message databases.


WCHOME Environment Variable
---------------------------

Although not required, you may specify your main WildCat directory
using the WCHOME environment variable.  While this will not save
any noticeable time, it may serve to avoid confusion especially if
you have more than one copy of MAKEWILD.DAT or CONFDESC.DAT on your
disk.  The following is an example:

     SET WCHOME=C:\WC30


Log File
--------

DupMsg writes to a log file which records information as specified
in the config file.  The default log file is DUPMSG.LOG, but like
the default config file it can be changed by entering a number on
the command line.  The section on command line options covers this
option in more detail.

If the log file is not found in the current directory or in the
path, a new log file will be created in the current directory.  If
a log file is found, DupMsg will add log information to the end of
it.


Command Line Options
--------------------

There are five optional command line parameters.  These are:

SETUP:    Runs the setup program to create and/or modify a config
          file.

RESET:    Resets the config file high message numbers to zero.

HIGH:     Sets the config file high message numbers to match the
          database high message number.  This is useful after a
          database renumber.

HELP:     Display the DupMsg help screen.
1-99:     An optional number specifies which configuration and log
          files to use.  If a number is specified, the default
          config file  "DUPMSG.CFG" is not used.  Instead, a config
          file numbered DUPMSG1.CFG to DUPMSG99.CFG will be used.
          For example:

          DUPMSG 15

          will tell DupMsg to use config file 'DUPMSG15.CFG' and
          log file 'DUPMSG15.LOG'.

If no command line parameters are used, DupMsg will use the values
in DUPMSG.CFG to run the DupMsg program and log information to
DUPMSG.LOG.


DOS Errorlevel
--------------

DupMsg returns one of three DOS errorlevels at the end of the
program.  These errorlevels can be tested in a batch file to
perform special functions (like deleting the log file) if desired.

Errorlevel 2:  DupMsg ended abnormally.  A required file may be
               missing, there may be memory problems, or perhaps
               there was a database error.  See the log file for
               more details.

Errorlevel 1:  DupMsg successfully deleted some duplicate
               messages.  If you are interested in which
               duplicates were found, you can use the batch file
               to save the log for later reading.

Errorlevel 0:  DupMsg ended successfully but did not find any
               duplicates, though some old or marked messages may
               have been deleted.

Batch files can be setup to test for errorlevels as follows:

     If Errorlevel = 2  .......
     If Errorlevel = 1  .......
     If Errorlevel = 0  .......

If the errorlevel test is true, whatever follows it will be treated
like a regular DOS command.  There are some exceptions, so refer to
a good DOS reference for assistance.

Remember that if errorlevel 2 is true, both of the following lines
(errorlevel 1 & 0) will also be true.  Batch files can be
programmed to skip lines by using GOTO statements, such as:

     If Errorlevel = 2 goto ERROR
     If Errorlevel = 1 goto SAVE
     If Errorlevel = 0 goto DELETE
     :ERROR
          {Perform error processing}
     Goto END

     :SAVE
          {Save the log file for later reading}
     Goto END

     :DELETE
          {Delete the log file}

     :END


Registration & Support
----------------------

DupMsg is distributed as shareware.  Shareware software allows you
to try before you buy, thus you never need to pay for a shareware
program that does not meet your needs.

You may use DupMsg for 30 days at no charge, allowing you a chance
to see if DupMsg meets your needs.  If you keep using DupMsg after
30 days, you must register the software by sending in the
registration form and paying the registration fee of $25 (US).
Should you forget to register, DupMsg will start to display a
reminder on the screen and wait a few seconds before continuing.

Registration entitles the purchaser to use DupMsg on one BBS
system.  A BBS system is defined as either a single computer
system, or a network system where multiple computers are connected
by direct cable to the same network server.

Even after you register DupMsg, your satisfaction is guaranteed.
If within 30 days after registering DupMsg, you decide that you no
longer wish to use the program, simply write a letter to the author
requesting a refund and your registration fee will be refunded.
Only the registration fee will be refunded, not the cost of
diskettes that may have been ordered.

The author can be reached via the WildNet, FidoNet, and MSInet mail
networks;  Address messages to Jim Metzler.

Registration fees and requests for additional information should be
addressed to:

Jim Metzler
WPI Box 188
100 Institute Rd.
Worcester, MA 01609

Voice: [phone]
BBS:   [phone]


Distribution
------------
Several WildNet Sysops have agreed to post DupMsg in its original
ARJ archive with Security Envelope.  Thanks go to these Sysops and
the many others who have supported DupMsg.

The original archive may be downloaded from the following BBSs:

Miwok Village        --  Worcester, MA         --  [phone]
MSI HQ BBS           --  Bakersfield, CA       --  [phone]

B-Line BBS           --  Bakersfield, CA       --  [phone]
Bun's BBS            --  San Francisco, CA     --  [phone]
Lost BBS             --  Overland Park, KS     --  [phone]
NoHo BBS             --  Hesperia, CA          --  [phone]
Programmer's Forum   --  Statesville, NC       --  [phone]

Data Base Forum      --  Oslo, Norway          --  [phone]
Gil's PC             --  Chapeau, Quebec       --  [phone]


DupMsg, Censor and the Enhanced UTIs can be F'Reqed from:

    Programmer's Forum  --  HST Dual v.32bis   --  (1:379/1102)
    Lost BBS            --  Compucom, V.32     --  (1:280/319)
    B-Line BBS          --  HST Dual v.32bis   --  (1:210/46)

The request names are DUPMSG, CENSOR and UTI.


You are authorized to distribute DupMsg provided the executable
program, documentation, and registration form are included in their
original, unmodified state.  No fee may be charged for the
distribution of DupMsg without the express written permission of
the author.

Please do not compress the executable program; testing has shown
that DupMsg takes approximately 10% longer to perform a given task
when compressed versus its original, un-compressed state.


Warranty Disclosure
-------------------

Normally, software distributed through bulletin boards systems or
shared from one user to another does not carry a warranty, mainly
because the author does not have control over the product and the
program can be modified either intentionally or accidentally
without the author's knowledge.

Recognizing this, DupMsg carries either no warranty or a limited
warranty depending on the distribution method.

LIMITED WARRANTY:

     This applies only when the program is received directly from
     the author on diskette, or when it is received in the original
     ARJ archive, complete with a valid Security Envelope.  The ARJ
     Security Envelope must read "ARJ archive created by James W.
     Metzler R#0301".  The user must keep the diskette or original
     ARJ archive as proof of this warranty.

     What is covered:  The DupMsg program will perform
     substantially as described in this document.  In addition,
     while the program may contain minor bugs or errors, it is
     warranted to be free of major defects for a period of six
     months.

     What is not covered:  The author will not be liable for
     incidental and/or consequential damages, including injury to
     property, interruption of service, loss of business and/or
     anticipated profits, or other consequential damages that may
     result from use, malfunction or failure of the DupMsg program,
     or from errors or omissions in the documentation.  Some states
     do not allow the exclusion or limitation of incidental or
     consequential damages, so the above limitation may not apply
     to you.

     Any attempt to modify the DupMsg program will void this
     warranty in its entirety.

     If there is a problem:  Within 60 days of notification that a
     major defect exists, the author, at his discretion, will
     either fix the program and forward the replacement at no cost
     to the registered user, or will refund the registration fee.

     How to get service:  Send a written letter to the author
     detailing the type of problem and symptoms experienced.
     Include your name, address, voice phone number and
     registration number and any additional information that you
     believe may be useful, such as any TSRs and disk cache
     programs in use.

     The author may contact you by voice or written letter for more
     information to help in tracking down the problem.


NO WARRANTY:

     If you did not receive the DupMsg program directly from the
     author, or in its original ARJ archive with a genuine Security
     Envelope (see above), then there is a possibility that the
     program and/or documentation has been modified, or that the
     program is not the author's work.  The author makes no
     warranty of any kind, express or implied, including any
     warranties of fitness and/or merchantability if the program
     was received from other sources.

     The author will not be liable for any damages, whether direct,
     indirect, special or consequential arising from a failure of
     this program to operate in the manner desired by the user.
     The author will not be liable for any damage to data or
     property which may be caused directly or indirectly by use of
     the program.

When trying new software, always backup any and all important files
on your system.


Copyright
---------

This program is:
Copyright (c) 1991, 1993 James W. Metzler, All Rights Reserved.